Contextual Representation in NLP to Improve Success in Accident Classification of Mine Safety Narratives

Contextual representation has taken center stage in Natural Language Processing (NLP) the recent past. Models such as Bidirectional Encoder Representations from Transformers (BERT) have found tremendous success arena. As a first attempt mining industry, current work, BERT architecture is adapted developing MineBERT model to accomplish classification of accident narratives US Mine Safety and Health Administration (MSHA) data set. In past multi-year research, several machine learning (ML) methods were used by authors improve rates nine significant MSHA categories. Out nine, for four major categories (“Type Groups”) five “narrow groups”, Random Forests (RF) registered 75% 42% rates, respectively, on average, while keeping false positives under 5%. Feature-based innovative NLP accident-specific expert choice vocabulary (ASECV) similarity score (SS) developed upon RF rates. A combination all these (“Stacked” approach) able slightly over (71%) 73.28% category “Caught-in”. Homographs are identified problem that was preventing further success. Their presence creating ambiguity problems algorithms. Adaptation effectively solved problem. When compared RF, implementation improved among narrow groups 13% 32%, 1%, which very significant. However, unique technical aspects jargon, brought set challenges terms preparation data, selection hyperparameters, fine-tuning achieve best performance, met research.